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ABSTRACT 

Assessment through new technology has gained a firm foothold within the university system in 
the last decade. This paper summarizes the experiences during the introduction of digital 
examination over the past two years at the Royal Institute of Technology in Stockholm, Sweden. 
These experiences come from three different perspectives; teachers, students and 
administrators. From the teachers’ perspective the experience has been very positive - less time 
was allocated to grading written examinations, the grades are perceived as fairer and the time 
saved can be spent on increasing the quality of other parts of the courses. From a student 
perspective the experience has also been very positive - most students are enjoying obtaining 
the results much more quickly, editing their answers on the examination more easily and the 
grades are perceived as fairer. The administrators’ experience is far more complex. While some 
parts of the administrative system encouraged the introduction of digital examination, other parts 
tried to stop it. The paper concludes with some advice on implementing changes in written 
examinations, based on the experience of the Swedish case. 
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INTRODUCTION 

The Swedish higher educational system is under pressure. Competition from international 
universities is becoming fiercer, the expansion of massive open online courses and demographic 
projections indicate that the number of students entering the university system is peaking right 
now (cf. Breslow et al. 2013; Mirriahi et al. 2015). At the same time, financial support from the 
government is decreasing and universities are trying to locate new revenue streams. In order to 
stay competitive, Swedish universities are trying to develop strategies for using these scarce 
resources in the optimal way. The challenge is to strike a balance between quality and efficiency 
in education (Kaiser et al. 2014; Njoku 2015). 

Over the past ten years much improvement has been made within the Royal Institute of 
Technology regarding quality and innovation in education. The numbers of pedagogical courses 
that are offered to the teaching staff within the university has multiplied, and these are now 
mandatory for teaching staff. Furthermore, the merits of educational experience have been 
emphasized in hiring new members of staff. Even though problem-based learning, constructive 
alignment and peer instruction are common in most courses today, some other aspects of 
education and learning remain the same (cf. Biggs & Tang 2007). One such conservative trait has 
to do with written examinations. 

Since the founding of the university in 1827, all written examinations have been carried out using 
paper and pen. The problems with using paper and pen are numerous and include problems with 
reading and grading because of poor handwriting, distribution of examinations between teachers 
in the same course, written exams getting lost. In comparison with other parts of the educational 
system, little has happened to improve quality and efficiency when it comes to written 
examinations (cf. Okonkwo 2010; Albugami & Ahmed 2015). 
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During 2013 and 2014 a number of teachers initiated a project to increase efficiency in written 
examinations. There was a general understanding that the final part of the courses consumed too 
much time and effort in relation to the other parts. On average, teachers spent 20 percent of the 
allocated course time in grading examinations. Hence, newly developed software for digital 
examinations was identified and a license purchased. The purpose of this paper is to describe 
and analyse the implementation of this new assessment system. 


THEORETICAL POINTS OF DEPARTURE 
Professionalism vs bureaucracy 

Over fifty years ago, there was a growing awareness that any university was home to two 
different ways of thinking: bureaucracy and professionalism, and that these two systems would at 
times conflict (Etzioni 1959). Among other things, it was shown that the experience of conflict 
between the two followed a U-shaped curve: universities with a high degree of differentiation or a 
very low degree of differentiation experienced less conflict. A high degree of differentiation would 
lead to a separation between the professionals and the bureaucrats, while on the other hand a 
high integration of the two would lead to a situation where both were continuously involved in 
daily operations, working in close cooperation (Darkenwald 1971). This description of the 
harmonious coexistence of bureaucracy and professionalism is based on an ideal equilibrium. 
However, at times, most organizations need to change in order to survive in changing institutional 
environments (Drew 2010). Change is perceived very differently in the two systems. 

Legitimacy of change in the two systems 

Somewhat simplified, the professional organization would be attuned to professional issues of 
teaching and research, based on a peer system where impulses for change can arise anywhere 
among grass-root peers (Blackmore & Sachs 2000). In comparison, bureaucracy is based on a 
strictly defined hierarchy, where impulses for change should adhere to the same command 
chains as every other strategic initiative. This means that among professionals, an impulse for 
change should originate among peers in order to be perceived as legitimate, whereas in the 
bureaucracy, the impulse for change should originate among top management, in order to be 
perceived as legitimate (Ramsden 1998; Macfarlane 2015). 

Some changes do originate in the institutional environment, such as in new directives from the 
government, or new legislation. These changes would be championed by central university 
administration, and expected to be implemented down through the hierarchy (Ginsberg 2011). 
Other changes originate in the daily teaching or research work, and such changes would be 
championed by the peers who discovered or invented them, and then propagate laterally around 
the university through professional networks, unless they were discovered by university top 
management and adopted by them (Wei et ai. 2014). The changes that have the best chance of 
success in a university context would be those where the interests of top management and the 
professional peers coincide, regardless of where the idea for change originated. Such changes 
would activate incentives found in both systems (Wedell 2009). 

Incentives for change in the two systems 

The incentives that motivate employees in the two systems differ markedly. For professionals, 
performance is based on the professional ideals such as pursuit of knowledge, democratic ideals, 
academic performance, intellectualism, and a system of peers within the profession (Deem & 
Brehony 2005). Any performance within this system is judged in terms of pedagogy or research 
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results (Winter 2009). As long as the results are successful, there is plenty of room for 
improvisation, intuition, creativity and eccentricity in both teaching and research. This means that 
teaching and research can be either conservative or progressive, depending on the specific 
context. 

In the bureaucratic system, work is specified in terms of procedures, and must follow these 
procedures impersonally (Breton & Wintrobe 2008). The end result of a bureaucratic procedure is 
not as important as following the procedure in detail. There is no room for improvisation, 
creativity, intuition or individuality/eccentricity in a bureaucratic system. This means that the 
bureaucratic system is inherently conservative - errors and deviations are more dangerous than 
any potential improvements (cf. Kahneman & Tversky 1979; Parkinson 1957; Evoh 2007). 

Thus, the only time incentives for change overlap between the two systems is when a change is 
accepted by the professionals as well as championed by university top management. 


THE STUDY 

The Royal Institute of Technology 

The Royal Institute of Technology is Sweden's leading technical university and is situated in the 
capital of Sweden, Stockholm. The university employs 4 000 teachers, researchers and 
administrators and approximately 14 000 students are enrolled in different programs. The majority 
of students are studying engineering degrees and a substantial number are international 
students, especially in the master’s programmes. 

The Royal Institute of Technology is organized in ten schools, from the School of Industrial 
Engineering and Management to the School of Computer Science and Communication. The 
schools have some freedom in developing their own strategies and using funds, even though 
centralization has increased during the last couple of years, especially when it comes to 
compliance with laws and regulations (cf. Ginsberg 2011). For instance, in the past five years the 
cost of administration has risen more than 30 percent. 

The study presented in this paper emanates from the Department of Real Estate and 
Construction management within the School of Architecture and the Built Environment, the 
second largest school of the university. It is responsible for some of the most popular and sought 
after programmes within the Royal Institute of Technology. The top three most popular 
programmes as of 2014, in terms of applicants per admitted student, are the programmes in 
architecture, real estate and finance and real estate development with agency. This department is 
responsible for the management of a number of programmes at bachelor and master level. 

In the autumn of 2013 the programme director for two of the department's programmes was 
contacted by an administrator at the school level, who had been contacted by a Swedish firm that 
had developed new software for digital examination. Among its customers were two of the leading 
Swedish universities. The programme director was invited to continue the dialogue with the 
supplier if he deemed the software to be of interest. After two meetings with the supplier, the 
programme director decided to test the software owing to the perceived benefits it had in a 
number of dimensions. 

Empirical data 


The empirical data in this paper comes from a number of different sources. Multiple sources were 
used when the data was collected, as recommended by a number of earlier studies (Corley 2004; 
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Gioia ef al. 2012) to address the issues of reliability and validity. Firstly and most importantly, 
students' course evaluations from courses involving digital examination were used. According to 
Swedish regulations, these evaluations are mandatory. Secondly, ten teachers who used digital 
examinations were interviewed. These interviews took around 30 minutes to one hour to 
complete. The interviews were semi-structured and centred on the perceived benefits and 
drawbacks of digital examination (Wengraf 2001; Diefenbach 2009). Thirdly, negotiations and 
meetings with representatives from the central university administration and local school 
administration were attended. We have also interviewed two local administrators. 

By using three different sources of data, it is believed that the results of this study represent 
important insights into the consequences of an implementation of this type in a university setting 
(Kumar et al. 1993; Gioia et al. 2010). 


RESULTS FROM THE STUDY 
Stage 1 - Pilot trial 

During the spring semester of 2014 a live test of the examination system was undertaken on a 
smaller course with only 28 students. The course had two midterm examinations and one final 
examination and all students took the examinations on their laptops. Two students who did not 
have a computer for the examination were supplied one by the head teacher in the course. 

In the course evaluation, students were asked whether the department should continue with 
digital examination. Of the 28 students, 26 were very positive and expressed the view that digital 
examinations should be the new standard while 2 indicated that the decision should be taken by 
the head teachers. None of the student preferred the traditional, written examination. The 
comments left by the students indicated that the overwhelming majority of the students were of 
the opinion that the programme director should change all written examinations to digital 
examinations throughout the entire programme. The remaining comments could be summarized 
by the comment: 

“Digital examination is the best thing that has happened”, anonymous student. 

From the teachers’ point of view, the experience was overwhelmingly positive. The main 
advantage was the time saved during the assessment and grading of the examinations. Even 
during this pilot trial, when all teachers needed to learn the software, time was spent on the 
examination by teachers was the same as in the old system. In the old system it usually took 
around one day to grade all midterm examinations but with digital examinations the time was 
reduced to less than half a day. For instance, when the midterm examination was in the morning, 
the students obtained their result the same day, since the teachers could grade the examination 
the same day, there being no need to wait for administration to sort, copy and distribute the 
examination papers. 

Stage 2 - Going live 

The experiences from the pilot trial were shared among the colleagues on the department during 
a teacher’s conference and there was great interest among those who hadn’t tried digital 
examination. The main reason for not introducing digital examination, among the teachers who 
hadn’t tried it, was a perceived lack of time and lack of planning. 

Up to date, 31 examinations have been offered digitally for 1138 students, involving 22 teachers. 
Some students have taken only one course with digital examination while others have taken up to 
five examinations. Since it is not mandatory to have a laptop to study at the Royal Institute of 
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Technology, exams are not mandatory, but close to 90% of students chose the digital 
examination. 

For the total number of examinations taken at the department during the autumn, since the 
courses differ in number of participants, 65% of the examinations were digital, while 27% of the 
students were in courses which didn't offer digital examination. 9% of the students chose the 
traditional exam even if a digital examination was offered. 

It is believed that, with time, most of these courses will offer a choice between traditional and 
digital examinations. For the 9% of students that choose to take the traditional examination, the 
head teachers asked those students during the course evaluations why they chose the traditional 
examination. A few of them preferred a traditional examination but most said either they didn’t 
have a computer they could bring at the moment or that they had forgotten the computer on the 
very morning of the examination. 


ANALYSIS OF THE RESULTS 
The students’ perspective 

Of the perspectives we have analysed in this paper, students are probably the most positive 
towards the change from paper and pen to digital examination (cf. Bertheussen 2014). From the 
analysis of the comments in the course evaluations, the pattern is very clear. So far, the small 
number negative comments on digital examination have been constructive suggestions for 
improvement of the software. Once the developers received the suggestion, they promised to 
implement them. Some comments were from students using the system for the first time: they 
were worried about trusting the system. 

The advantages of the new system have been highlighted through the comments on the course 
evaluations, but also in dialogue with representatives from the students' unions. In short the major 
advantages with digital examination are the following: 

• Faster grading and quicker feedback. On average, the time spent on grading was cut in 
half, which meant that the students received more timely feedback on their examinations. 
The importance of quick feedback from a learning perspective has been highlighted by 
other researchers (see, Biggs & Tang 2007; Mostert & Quinn 2009). 

• Less stress. As the students were able to edit their answers using software instead of pen 
and eraser, they experienced much less stress during the examinations. With traditional 
examination, cramp in the hands and wrist pains are quite common. 

• Anonymity. A number of students felt that the fact that teachers were unable to identify 
students either through their name or hand-writing rendered the examinations fairer. 

To summarize, negative comments were extremely few and very much on the constructive side. 
In all, the students felt that the incentives of taking the digital examination were great in 
comparison with traditional paper and pen examination. Or in the words of one of our students: 

“Without doubt, digital examinations are the best that has happened to assessments. I 
like the fact that you can feel 100% anonymous during the grading, not only regarding 
date of birth but also handwriting. I also like that you may proofread your answers and 
change typos or errors easy. This way you won’t lose any points on carelessness”. 

The teachers’ perspective 

The teachers' perspective of the changes is the one that is closest to programme implementers, 
as change agents involved in the planning and implementation of the digital examinations. Even 
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though implementers were positive after the first trials, the speed of the change and the interest 
from fellow teachers was underestimated. The positive results from using digital examinations 
emanating from the discussions with the teachers and implementers’ experiences can be 
summarized in the following points: 

• A quicker grading process. Most teachers estimated that using digital examination saved 
them at least 50% of the time traditionally allocated to grading. The saved time could be 
used for other things, such as course development. 

• A fairer system for grading. Since the time it takes to grade examinations is shortened, 
the fatigue is reduced which increases the likelihood for fair grading (cf. Biggs & Tang 
2007; Alomari 2009). Furthermore, many teachers indicated that students could no longer 
hide behind poor handwriting, which also is important for fair grading. 

• Less risk of losing examination papers. From time to time, examinations have been lost 
or mixed up. With the digital examinations, there are no longer any physical copies of the 
exams to lose hence this risk is mitigated. 

The incentives for changing from traditional to digital examinations are very concrete and the 
negative effects from the teachers' perspective are slight if not non-existent. The only issue 
highlighted in the interviews was the time it took to learn a new system. Apart from this, 
comments were very positive. 

The administrators’ perspective 

The administrators have a number of important roles within the university system. At the Royal 
Institute of Technology, administrators can be found at three different levels: the local level in the 
department, the school level and at central headquarters. 

The administrators at the local level were very positive about the introduction of digital 
examinations. The primary advantage for them was the reduced physical handling of examination 
papers, such as copying, and the reduced time they have to spend on these issues. 

The administrators at the school level have less to do with the daily operations and meeting with 
students. Therefore they had more difficulty grasping the positive aspects of the introduction of 
digital examinations. It took a number of meetings with them in which a number of risks with the 
software were discussed as well as compliance issues. Budgeting, the cost of the software and 
other less immediate issues tended to dominate these meetings. 

Finally, there have been a number of meetings with administrators from the central headquarters 
of the university. Since they are furthest from the daily operations of teaching and learning at the 
grass root level, the positive effects that both students and teachers experienced with digital 
examination and the positive effects it had on course design were not apparent for them. Neither 
was this a reasonable and relevant argument in the discussion. The overarching goal for central 
administrators is to comply with the legal framework within the university system. They are not 
incentivized by more satisfied students or teachers who are enjoying their work. This is more or 
less a non-issue for them. It should not have been a major surprise that central administrators 
tried to stop digital examinations. Furthermore, one year earlier, against the advice of teachers, 
they had decided to scan all written examinations, which represented an enormous investment for 
the university. The introduction of a cheaper and quicker solution from the grass-roots was not 
popular (cf. Ginsberg 2011). 
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CONCLUDING REMARKS 

There are a number of important findings in this study that could be applied to other settings. First 
of all, we do recommend other teachers to look for alternatives to the traditional paper and pen 
examination, used in most universities worldwide. The advantages are enormous, from both a 
pedagogical and economical perspective. 

Given the importance of sustainability, there are obvious and large benefits in using digital 
examinations over traditional examinations. Every year universities worldwide spend a lot of 
resources on exams. Digital examinations will effectively eliminate much spending. 

The major obstacle to the introduction of digital examinations can be found at the central 
administration level. Our recommendation would be to find ways of engaging them at an early 
stage of the process and to find incentives that they perceive as rewarding and aligned with their 
mission. This is not an easy undertaking, but it can be done (cf. Ginsberg 2011). 
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